Translating fpp3 Chapter 2 examples from R to Clojure using tablecloth, tablecloth.time, and tableplot.
Reference: https://otexts.com/fpp3/graphics.html
Run with: clj -A:notebooks then evaluate in your editor, or render with Clay: (clay/make! {:source-path "notebooks/chapter_02_time_series_graphics.clj"})
(ns chapter-02-time-series-graphics
(:require [tablecloth.api :as tc]
[tablecloth.column.api :as tcc]
[tech.v3.datatype :as dtype]
[tech.v3.datatype.functional :as dfn]
[tech.v3.dataset :as ds]
[scicloj.tableplot.v1.plotly :as plotly]
[scicloj.kindly.v4.kind :as kind]
[clojure.data.json :as json]
[tablecloth.time.api :as time-api]
[tablecloth.time.column.api :as time-col])
(:import [java.time LocalDate]))In R, fpp3 provides datasets as tsibble objects with declared index and keys. In Clojure, we use plain tablecloth datasets loaded from CSV. The time column is just a column — no special metadata needed.
(defn load-fpp3
"Load one of the fpp3 datasets from CSV."
[name]
(tc/dataset (str "data/fpp3/" name ".csv")))R: PBS (67,596 × 9, monthly, keyed by Concession/Type/ATC1/ATC2)
(def PBS (load-fpp3 "PBS"))PBSdata/fpp3/PBS.csv [67596 9]:
| Month | Concession | Type | ATC1 | ATC1_desc | ATC2 | ATC2_desc | Scripts | Cost |
|---|---|---|---|---|---|---|---|---|
| 1991-07-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 18228.0 | 67877.00 |
| 1991-08-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 15327.0 | 57011.00 |
| 1991-09-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 14775.0 | 55020.00 |
| 1991-10-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 15380.0 | 57222.00 |
| 1991-11-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 14371.0 | 52120.00 |
| 1991-12-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 15028.0 | 54299.00 |
| 1992-01-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 11040.0 | 39753.00 |
| 1992-02-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 15165.0 | 54405.00 |
| 1992-03-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 16898.0 | 61108.00 |
| 1992-04-01 | Concessional | Co-payments | A | Alimentary tract and metabolism | A01 | STOMATOLOGICAL PREPARATIONS | 18141.0 | 65356.00 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2007-08-01 | General | Safety net | Z | Z | Z | 250.0 | 3030.15 | |
| 2007-09-01 | General | Safety net | Z | Z | Z | 281.0 | 3822.94 | |
| 2007-10-01 | General | Safety net | Z | Z | Z | 374.0 | 4346.34 | |
| 2007-11-01 | General | Safety net | Z | Z | Z | 501.0 | 6930.19 | |
| 2007-12-01 | General | Safety net | Z | Z | Z | 655.0 | 7939.00 | |
| 2008-01-01 | General | Safety net | Z | Z | Z | 797.0 | 9604.00 | |
| 2008-02-01 | General | Safety net | Z | Z | Z | 135.0 | 1591.00 | |
| 2008-03-01 | General | Safety net | Z | Z | Z | 15.0 | 276.00 | |
| 2008-04-01 | General | Safety net | Z | Z | Z | 11.0 | 165.00 | |
| 2008-05-01 | General | Safety net | Z | Z | Z | 21.0 | 278.00 | |
| 2008-06-01 | General | Safety net | Z | Z | Z | 57.0 | 491.00 |
In R:
PBS |> filter(ATC2 == "A10") |> select(Month, Concession, Type, Cost) |> summarise(TotalC = sum(Cost)) |> mutate(Cost = TotalC / 1e6) -> a10
(def a10
(-> PBS
(tc/select-rows #(= "A10" (get % "ATC2")))
(tc/select-columns ["Month" "Concession" "Type" "Cost"])
(tc/group-by ["Month"])
(tc/aggregate {"TotalC" #(dfn/sum (% "Cost"))})
(tc/add-column "Cost" #(dfn// (% "TotalC") 1e6))))a10_unnamed [204 3]:
| Month | TotalC | Cost |
|---|---|---|
| 1991-07-01 | 3.52659100E+06 | 3.52659100 |
| 1991-08-01 | 3.18089100E+06 | 3.18089100 |
| 1991-09-01 | 3.25222100E+06 | 3.25222100 |
| 1991-10-01 | 3.61100300E+06 | 3.61100300 |
| 1991-11-01 | 3.56586900E+06 | 3.56586900 |
| 1991-12-01 | 4.30637100E+06 | 4.30637100 |
| 1992-01-01 | 5.08833500E+06 | 5.08833500 |
| 1992-02-01 | 2.81452000E+06 | 2.81452000 |
| 1992-03-01 | 2.98581100E+06 | 2.98581100 |
| 1992-04-01 | 3.20478000E+06 | 3.20478000 |
| ... | ... | ... |
| 2007-08-01 | 2.39302035E+07 | 23.93020353 |
| 2007-09-01 | 2.29303569E+07 | 22.93035694 |
| 2007-10-01 | 2.32633399E+07 | 23.26333992 |
| 2007-11-01 | 2.52500302E+07 | 25.25003022 |
| 2007-12-01 | 2.58060900E+07 | 25.80609000 |
| 2008-01-01 | 2.96653560E+07 | 29.66535600 |
| 2008-02-01 | 2.16542850E+07 | 21.65428500 |
| 2008-03-01 | 1.82649450E+07 | 18.26494500 |
| 2008-04-01 | 2.31076770E+07 | 23.10767700 |
| 2008-05-01 | 2.29125100E+07 | 22.91251000 |
| 2008-06-01 | 1.94317400E+07 | 19.43174000 |
(def ansett (load-fpp3 "ansett"))ansettdata/fpp3/ansett.csv [7407 4]:
| Week | Airports | Class | Passengers |
|---|---|---|---|
| 1989-07-10 | ADL-PER | Business | 193.0 |
| 1989-07-17 | ADL-PER | Business | 254.0 |
| 1989-07-24 | ADL-PER | Business | 185.0 |
| 1989-07-31 | ADL-PER | Business | 254.0 |
| 1989-08-07 | ADL-PER | Business | 191.0 |
| 1989-08-14 | ADL-PER | Business | 136.0 |
| 1989-08-21 | ADL-PER | Business | 0.0 |
| 1989-08-28 | ADL-PER | Business | 0.0 |
| 1989-09-04 | ADL-PER | Business | 0.0 |
| 1989-09-11 | ADL-PER | Business | 0.0 |
| ... | ... | ... | ... |
| 1992-09-07 | SYD-PER | First | 176.0 |
| 1992-09-14 | SYD-PER | First | 191.0 |
| 1992-09-21 | SYD-PER | First | 204.0 |
| 1992-09-28 | SYD-PER | First | 183.0 |
| 1992-10-05 | SYD-PER | First | 220.0 |
| 1992-10-12 | SYD-PER | First | 234.0 |
| 1992-10-19 | SYD-PER | First | 203.0 |
| 1992-10-26 | SYD-PER | First | 137.0 |
| 1992-11-02 | SYD-PER | First | 161.0 |
| 1992-11-09 | SYD-PER | First | 155.0 |
| 1992-11-16 | SYD-PER | First | 188.0 |
(def aus-production (load-fpp3 "aus_production"))aus-productiondata/fpp3/aus_production.csv [218 7]:
| Quarter | Beer | Tobacco | Bricks | Cement | Electricity | Gas |
|---|---|---|---|---|---|---|
| 1956-01-01 | 284.0 | 5225.0 | 189.0 | 465.0 | 3923.0 | 5.0 |
| 1956-04-01 | 213.0 | 5178.0 | 204.0 | 532.0 | 4436.0 | 6.0 |
| 1956-07-01 | 227.0 | 5297.0 | 208.0 | 561.0 | 4806.0 | 7.0 |
| 1956-10-01 | 308.0 | 5681.0 | 197.0 | 570.0 | 4418.0 | 6.0 |
| 1957-01-01 | 262.0 | 5577.0 | 187.0 | 529.0 | 4339.0 | 5.0 |
| 1957-04-01 | 228.0 | 5651.0 | 214.0 | 604.0 | 4811.0 | 7.0 |
| 1957-07-01 | 236.0 | 5317.0 | 227.0 | 603.0 | 5259.0 | 7.0 |
| 1957-10-01 | 320.0 | 6152.0 | 222.0 | 582.0 | 4735.0 | 6.0 |
| 1958-01-01 | 272.0 | 5758.0 | 199.0 | 554.0 | 4608.0 | 5.0 |
| 1958-04-01 | 233.0 | 5641.0 | 229.0 | 620.0 | 5196.0 | 7.0 |
| ... | ... | ... | ... | ... | ... | ... |
| 2007-10-01 | 473.0 | 2562.0 | 56411.0 | 205.0 | ||
| 2008-01-01 | 420.0 | 2183.0 | 59118.0 | 194.0 | ||
| 2008-04-01 | 390.0 | 2558.0 | 56660.0 | 229.0 | ||
| 2008-07-01 | 410.0 | 2612.0 | 64067.0 | 249.0 | ||
| 2008-10-01 | 488.0 | 2373.0 | 59045.0 | 203.0 | ||
| 2009-01-01 | 415.0 | 1963.0 | 58368.0 | 196.0 | ||
| 2009-04-01 | 398.0 | 2160.0 | 57471.0 | 238.0 | ||
| 2009-07-01 | 419.0 | 2325.0 | 58394.0 | 252.0 | ||
| 2009-10-01 | 488.0 | 2273.0 | 57336.0 | 210.0 | ||
| 2010-01-01 | 414.0 | 1904.0 | 58309.0 | 205.0 | ||
| 2010-04-01 | 374.0 | 2401.0 | 58041.0 | 236.0 |
Time column is "2011-12-31 13:00:00" — not auto-parsed by tablecloth. Use tc/convert-types with a format pattern to parse it.
(def vic-elec
(-> (load-fpp3 "vic_elec")
(tc/convert-types "Time" [:local-date-time "yyyy-MM-dd HH:mm:ss"])))vic-elecdata/fpp3/vic_elec.csv [52608 5]:
| Time | Demand | Temperature | Date | Holiday |
|---|---|---|---|---|
| 2011-12-31T13:00 | 4382.825174 | 21.40 | 2012-01-01 | True |
| 2011-12-31T13:30 | 4263.365526 | 21.05 | 2012-01-01 | True |
| 2011-12-31T14:00 | 4048.966046 | 20.70 | 2012-01-01 | True |
| 2011-12-31T14:30 | 3877.563330 | 20.55 | 2012-01-01 | True |
| 2011-12-31T15:00 | 4036.229746 | 20.40 | 2012-01-01 | True |
| 2011-12-31T15:30 | 3865.597244 | 20.25 | 2012-01-01 | True |
| 2011-12-31T16:00 | 3694.097664 | 20.10 | 2012-01-01 | True |
| 2011-12-31T16:30 | 3561.623686 | 19.60 | 2012-01-01 | True |
| 2011-12-31T17:00 | 3433.035352 | 19.10 | 2012-01-01 | True |
| 2011-12-31T17:30 | 3359.468000 | 18.95 | 2012-01-01 | True |
| ... | ... | ... | ... | ... |
| 2014-12-31T07:30 | 4244.465530 | 23.40 | 2014-12-31 | False |
| 2014-12-31T08:00 | 4125.988252 | 22.30 | 2014-12-31 | False |
| 2014-12-31T08:30 | 4013.262848 | 20.90 | 2014-12-31 | False |
| 2014-12-31T09:00 | 3924.593434 | 20.30 | 2014-12-31 | False |
| 2014-12-31T09:30 | 3893.867974 | 20.30 | 2014-12-31 | False |
| 2014-12-31T10:00 | 3927.753088 | 20.30 | 2014-12-31 | False |
| 2014-12-31T10:30 | 3873.448714 | 19.00 | 2014-12-31 | False |
| 2014-12-31T11:00 | 3791.637322 | 18.50 | 2014-12-31 | False |
| 2014-12-31T11:30 | 3724.835666 | 17.70 | 2014-12-31 | False |
| 2014-12-31T12:00 | 3761.886854 | 17.30 | 2014-12-31 | False |
| 2014-12-31T12:30 | 3809.414586 | 17.10 | 2014-12-31 | False |
(def olympic-running (load-fpp3 "olympic_running"))olympic-runningdata/fpp3/olympic_running.csv [312 5]:
| column-0 | Year | Length | Sex | Time |
|---|---|---|---|---|
| 1 | 1896 | 100 | men | 12.00 |
| 2 | 1900 | 100 | men | 11.00 |
| 3 | 1904 | 100 | men | 11.00 |
| 4 | 1908 | 100 | men | 10.80 |
| 5 | 1912 | 100 | men | 10.80 |
| 6 | 1916 | 100 | men | |
| 7 | 1920 | 100 | men | 10.80 |
| 8 | 1924 | 100 | men | 10.60 |
| 9 | 1928 | 100 | men | 10.80 |
| 10 | 1932 | 100 | men | 10.30 |
| ... | ... | ... | ... | ... |
| 302 | 2008 | 10000 | men | 1621.17 |
| 303 | 2012 | 10000 | men | 1650.42 |
| 304 | 2016 | 10000 | men | 1625.17 |
| 305 | 1988 | 10000 | women | 1865.21 |
| 306 | 1992 | 10000 | women | 1866.02 |
| 307 | 1996 | 10000 | women | 1861.63 |
| 308 | 2000 | 10000 | women | 1817.49 |
| 309 | 2004 | 10000 | women | 1824.36 |
| 310 | 2008 | 10000 | women | 1794.66 |
| 311 | 2012 | 10000 | women | 1820.75 |
| 312 | 2016 | 10000 | women | 1757.45 |
R: autoplot(melsyd_economy, Passengers) Clojure: tableplot line chart
(def melsyd-economy
(-> ansett
(tc/select-rows #(and (= "MEL-SYD" (get % "Airports"))
(= "Economy" (get % "Class"))))
(tc/add-column "Passengers (000s)" #(dfn// (% "Passengers") 1000))))(-> melsyd-economy
(plotly/layer-line {:=x "Week"
:=y "Passengers (000s)"
:=title "Ansett airlines economy class: Melbourne-Sydney"}))Notable features visible in the plot:
(-> a10
(plotly/layer-line {:=x "Month"
:=y "Cost"
:=title "Australian antidiabetic drug sales"
:=y-title "$ (millions)"}))Clear increasing trend with strong seasonality that grows proportionally. The January spike each year is from stockpiling before year-end subsidies.
Three fundamental components:
The four examples from Figure 2.3:
(def recent-beer
(-> aus-production
(tc/select-rows #(>= (.getYear (get % "Quarter")) 2000))
(tc/select-columns ["Quarter" "Beer"])))(-> recent-beer
(plotly/layer-line {:=x "Quarter"
:=y "Beer"
:=title "Australian quarterly beer production"}))(def gafa (load-fpp3 "gafa_stock"))(def google-2015
(-> gafa
(tc/select-rows #(and (= "GOOG" (get % "Symbol"))
(= (.getYear (get % "Date")) 2015)))))Daily closing price
(-> google-2015
(plotly/layer-line {:=x "Date"
:=y "Close"
:=title "Google daily closing stock price (2015)"}))Daily change in closing price
(let [closes (vec (google-2015 "Close"))
diffs (mapv - (rest closes) (butlast closes))]
(-> (tc/dataset {"Date" (rest (vec (google-2015 "Date")))
"Change" diffs})
(plotly/layer-line {:=x "Date"
:=y "Change"
:=title "Google daily change in closing stock price (2015)"})))R: gg_season(a10, Cost) — overlay each year on the same month axis. We extract year and month, then plot with color = year.
tablecloth.time has add-time-columns — a dataset-level operation that extracts datetime fields in one call. tablecloth auto-parses our CSV date strings into :packed-local-date, so these work out of the box.
Vector form: column names match field names Map form: explicit output names
(def a10-seasonal
(-> a10
(time-api/add-time-columns "Month" {:year "Year" :month "MonthNum"})))Year is int64 — tableplot treats numeric columns as continuous color scales. Convert to string so it's treated as categorical (one line per year).
(-> a10-seasonal
(tc/add-column "YearStr" #(mapv str (% "Year")))
(plotly/layer-line {:=x "MonthNum"
:=y "Cost"
:=color "YearStr"
:=title "Seasonal plot: Antidiabetic drug sales"
:=x-title "Month"
:=y-title "$ (millions)"}))Each line is one year. The seasonal shape is clear:
Electricity demand has daily, weekly, and yearly patterns. R: gg_season(Demand, period = "day"|"week"|"year")
Now that Time is parsed as LocalDateTime, we can use add-time-columns directly. vic_elec is half-hourly, so we compute fractional hour (e.g. 13.5 for 13:30) to avoid duplicate x-values per hour. NOTE: The "Date" column is the billing/reporting date (next day), not the calendar date of the timestamp. We derive all groupings from the Time column.
(def vic-elec-with-fields
(-> vic-elec
(time-api/add-time-columns "Time" {:hour "Hour"
:minute "Minute"
:day "Day"
:month "Month"
:day-of-week "DayOfWeek"
:day-of-year "DayOfYear"
:week-of-year "WeekOfYear"
:year "Year"})
(tc/add-column "HourOfDay" #(dfn/+ (% "Hour") (dfn// (% "Minute") 60.0)))
;; Derive date string from Time (not the Date column)
(tc/add-column "TimeDate" #(mapv (fn [t] (str (.toLocalDate t))) (% "Time")))
(tc/add-column "YearStr" #(mapv str (% "Year")))
(tc/add-column "WeekLabel" #(mapv str (% "WeekOfYear")))
;; Proper week index for seasonal plots (ISO weeks cause cross-cutting lines)
(tc/add-column "WeekIndex" #(dfn// (dfn/- (% "DayOfYear") 1) 7))
;; Phase columns for seasonal plots
(tc/add-column "DailyPhase" #(dfn// (% "HourOfDay") 24.0))
(tc/add-column "WeeklyPhase" #(dfn// (dfn/+ (dfn/* (dfn/- (% "DayOfWeek") 1) 24) (% "HourOfDay")) 168.0))
;; Combined year-week for grouping in seasonal plots
(tc/add-column "YearWeek" #(mapv (fn [y w] (str y "-W" (format "%02d" (int w))))
(% "Year") (% "WeekIndex")))))Generate a Plotly spec for seasonal plots using tableplot as the base. This uses tableplot's layer-line with :=color to generate multiple traces, then post-processes to hide legend and set custom colors.
(defn seasonal-plot-spec
"Generate a Plotly spec for a seasonal plot.
- ds: dataset (should include phase column)
- phase-col: column for x-axis (phase within period, 0 to 1)
- value-col: column for y-axis
- group-col: column to group by (creates one trace per unique value)
- color-fn: fn from group-name (string) -> color string
Options:
- :line-width (default 0.3)
- :title, :x-title, :y-title for axis labels"
[ds phase-col value-col group-col color-fn
& {:keys [line-width title x-title y-title]
:or {line-width 0.3}}]
(let [;; Use tableplot to generate base spec with traces
viz (-> ds
(tc/order-by phase-col)
(plotly/layer-line {:=x phase-col
:=y value-col
:=color group-col
:=title title
:=x-title x-title
:=y-title y-title}))
;; Extract final Plotly spec
spec ((:kindly/f viz) viz)]
;; Post-process traces: hide legend, set colors
(update spec :data
#(mapv (fn [trace]
(-> trace
(assoc :showlegend false)
(assoc-in [:line :color] (color-fn (:name trace)))
(assoc-in [:line :width] line-width)))
%))))Daily pattern: phase = hour/24, each day overlaid. Using seasonal-plot-spec helper with tableplot as base.
(let [year-color #(get {"2011" "#7570b3" "2012" "#1b9e77" "2013" "#d95f02" "2014" "#7570b3"}
(subs % 0 4) "gray")]
(kind/plotly
(seasonal-plot-spec vic-elec-with-fields
"DailyPhase" "Demand" "TimeDate"
year-color
:title "Electricity demand: Victoria (daily pattern)"
:x-title "Phase of day (0=midnight, 0.5=noon)"
:y-title "MWh")))Weekly pattern: phase = hours_since_monday / 168, each week overlaid. Using seasonal-plot-spec helper with tableplot as base.
(let [year-color #(get {"2011" "#7570b3" "2012" "#1b9e77" "2013" "#d95f02" "2014" "#7570b3"}
(subs % 0 4) "gray")]
(kind/plotly
(seasonal-plot-spec vic-elec-with-fields
"WeeklyPhase" "Demand" "YearWeek"
year-color
:title "Electricity demand: Victoria (weekly pattern)"
:x-title "Phase of week (0=Mon, 0.5=Thu noon, 1=Sun midnight)"
:y-title "MWh")))Yearly pattern: x = day of year, each year is a line. All 3 years fit — only 3 traces.
(-> vic-elec-with-fields
(plotly/layer-line {:=x "DayOfYear"
:=y "Demand"
:=color "YearStr"
:=title "Electricity demand: Victoria (yearly pattern)"
:=x-title "Day of year"
:=y-title "MWh"}))R: gg_subseries(a10, Cost) — for each month, show values across years with the mean as a horizontal line.
Group by month, plot each month's values over years
(def a10-subseries
(-> a10
(time-api/add-time-columns "Month" {:year "Year" :month "MonthNum"})))Faceted by month — each panel shows that month across all years. Convert MonthNum to string so tableplot treats it as categorical.
(-> a10-subseries
(tc/add-column "MonthLabel" #(mapv str (% "MonthNum")))
(plotly/layer-line {:=x "Year"
:=y "Cost"
:=color "MonthLabel"
:=title "Seasonal subseries plot: Antidiabetic drug sales"
:=y-title "$ (millions)"}))R: ggplot(aes(x = Temperature, y = Demand)) + geom_point() Electricity demand vs temperature for 2014 Victoria.
(def vic-elec-2014
(-> vic-elec
(tc/select-rows #(= (.getYear (get % "Time")) 2014))))(-> vic-elec-2014
(plotly/layer-point {:=x "Temperature"
:=y "Demand"
:=title "Electricity demand vs temperature (Victoria, 2014)"
:=x-title "Temperature (°C)"
:=y-title "Demand (MWh)"}))The U-shape: high demand for both cold (heating) and hot (air conditioning). Correlation coefficient r = 0.28 — misleading for non-linear relationships. Always plot first!
(def tourism (load-fpp3 "tourism"))(def visitors-by-state
(-> tourism
(tc/group-by ["Quarter" "State"])
(tc/aggregate {"Trips" #(dfn/sum (% "Trips"))})
;; Pivot wider: one column per state
(tc/pivot->wider "State" "Trips")))visitors-by-state_unnamed [80 9]:
| Quarter | South Australia | Northern Territory | Western Australia | Victoria | New South Wales | Queensland | ACT | Tasmania |
|---|---|---|---|---|---|---|---|---|
| 1998-01-01 | 1735.4384181 | 181.4488234 | 1641.0894945 | 6010.4244905 | 8039.7947954 | 4041.3701591 | 551.0019215 | 981.6291663 |
| 1998-04-01 | 1394.6383194 | 313.9361515 | 1576.3265337 | 4795.2467552 | 7166.0138053 | 3967.9046526 | 416.0256229 | 693.2882267 |
| 1998-07-01 | 1213.3307229 | 528.4368592 | 1588.2936922 | 4316.8451697 | 6747.9357896 | 4593.8939910 | 436.0290111 | 401.8752750 |
| 1998-10-01 | 1452.5699692 | 247.7028173 | 1839.7169904 | 4674.8291177 | 7282.0823712 | 4202.8291407 | 449.7984449 | 680.6010392 |
| 1999-01-01 | 1541.1817910 | 184.8895923 | 1835.6875729 | 5304.3341954 | 7584.7768389 | 4332.4908503 | 378.5728168 | 925.4197220 |
| 1999-04-01 | 1636.1154606 | 366.0927858 | 1836.9642903 | 4561.7109009 | 7054.0387051 | 4824.4804518 | 558.1781421 | 620.7925480 |
| 1999-07-01 | 1282.9490168 | 501.4312841 | 1725.3748681 | 3783.6010393 | 6723.6771461 | 5018.0345042 | 448.9011959 | 430.2234538 |
| 1999-10-01 | 1386.9430724 | 248.4302996 | 1518.9426972 | 4201.4224779 | 7135.7669454 | 4349.8358766 | 594.8254416 | 591.7588297 |
| 2000-01-01 | 1832.8466955 | 206.0532200 | 1635.5250181 | 5566.8572876 | 7295.9866387 | 4413.2262339 | 599.6685405 | 789.1311444 |
| 2000-04-01 | 1415.4842932 | 359.7619246 | 1808.3286160 | 4501.9042822 | 6445.2201677 | 4344.1592770 | 557.1350565 | 509.0698648 |
| ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 2015-04-01 | 1561.2069129 | 533.0254161 | 2394.3740364 | 5284.4708775 | 7375.2085323 | 5555.8301970 | 516.8703428 | 577.9280518 |
| 2015-07-01 | 1383.9080928 | 682.5518237 | 2467.0642463 | 4981.1688294 | 6961.2502033 | 5866.0704509 | 688.2031884 | 455.5288199 |
| 2015-10-01 | 1700.3825231 | 423.2025164 | 2743.4490648 | 5550.8237034 | 7760.0190221 | 5532.2106436 | 597.2455694 | 832.8281787 |
| 2016-01-01 | 1871.4572503 | 352.0984958 | 2871.1718747 | 6599.7004830 | 8287.8720804 | 5042.1537503 | 625.1416344 | 1011.0421206 |
| 2016-04-01 | 1660.1266358 | 469.1365356 | 2469.3942809 | 5335.2297369 | 7660.5796206 | 5446.5536505 | 592.6084987 | 651.3987982 |
| 2016-07-01 | 1534.2342547 | 713.8207499 | 2317.0445437 | 5221.8805097 | 7326.3046707 | 5939.3346446 | 572.4370928 | 566.2636651 |
| 2016-10-01 | 1635.4677141 | 340.6959274 | 2656.3307006 | 6113.0164129 | 7995.5021920 | 6106.3838529 | 667.2141410 | 832.9900323 |
| 2017-01-01 | 1815.4121833 | 298.1660264 | 2570.9116892 | 7269.5270136 | 8320.7034623 | 5451.9871280 | 634.3687469 | 1135.3127709 |
| 2017-04-01 | 1660.1320376 | 621.4413922 | 2438.4879387 | 5901.3865473 | 8285.0262795 | 5638.4735354 | 748.2904309 | 820.3685463 |
| 2017-07-01 | 1514.7861308 | 597.6583969 | 2493.9549995 | 5817.9715437 | 8298.2574173 | 6533.8369322 | 631.7599043 | 618.0893828 |
| 2017-10-01 | 1869.1069854 | 346.0619384 | 2635.7542961 | 6865.3988511 | 8542.4906073 | 5813.9036668 | 720.3293701 | 800.5084986 |
For a scatterplot matrix, we'd plot each state column against every other. Tableplot doesn't have a built-in pairs plot, but you can compose them. Here's one pair as an example — NSW vs Victoria:
(-> visitors-by-state
(plotly/layer-point {:=x "New South Wales"
:=y "Victoria"
:=title "Tourism: NSW vs Victoria (quarterly trips)"}))R: gg_lag(Beer, geom = "point") Plot y_t against y_{t-k} for various lags. This is the visual precursor to autocorrelation.
For beer production, lag 4 should show strong positive correlation (seasonal) Using tablecloth.time.api/add-lags with auto-drop of missing values: Note: add-lags creates keyword columns like :Beer_lag4
(-> recent-beer
(time-api/add-lags "Beer" [4])
(plotly/layer-point {:=x "Beer_lag4"
:=y "Beer"
:=title "Lag 4 plot: Australian beer production"
:=x-title "Beer (t-4)"
:=y-title "Beer (t)"}))Strong positive diagonal = strong correlation at lag 4 (Q4 peaks align with Q4 peaks from the previous year)
r_k = Σ(y_t - ȳ)(y_{t-k} - ȳ) / Σ(y_t - ȳ)²
This is a core function we need in tablecloth.time. For now, let's compute it manually.
(defn acf
"Compute autocorrelation coefficients for lags 1..max-lag.
Returns a dataset with :lag and :acf columns."
[values max-lag]
(let [values (double-array (remove nil? values))
n (alength values)
mean (/ (areduce values i sum 0.0 (+ sum (aget values i))) n)
;; denominator: Σ(y_t - ȳ)²
denom (areduce values i sum 0.0
(let [d (- (aget values i) mean)]
(+ sum (* d d))))
lags (range 1 (inc max-lag))
acf-vals (mapv (fn [k]
(let [numer (loop [t k, sum 0.0]
(if (>= t n)
sum
(recur (inc t)
(+ sum (* (- (aget values t) mean)
(- (aget values (- t k)) mean))))))]
(/ numer denom)))
lags)]
(tc/dataset {"lag" (vec lags)
"acf" acf-vals})))(def beer-acf (acf (recent-beer "Beer") 9))beer-acf_unnamed [9 2]:
| lag | acf |
|---|---|
| 1 | -0.05298108 |
| 2 | -0.75817544 |
| 3 | -0.02623376 |
| 4 | 0.80220453 |
| 5 | -0.07747120 |
| 6 | -0.65745127 |
| 7 | 0.00119492 |
| 8 | 0.70725408 |
| 9 | -0.08875626 |
Should match R output: lag 1: -0.053, lag 2: -0.758, lag 4: 0.802, lag 8: 0.707
(let [T (count (remove nil? (vec (recent-beer "Beer"))))
bound (/ 1.96 (Math/sqrt T))]
(-> beer-acf
(tc/add-column "upper" (repeat (tc/row-count beer-acf) bound))
(tc/add-column "lower" (repeat (tc/row-count beer-acf) (- bound)))
(plotly/layer-bar {:=x "lag"
:=y "acf"
:=title "ACF: Australian beer production"
:=y-title "Autocorrelation"})))(def a10-acf (acf (a10 "Cost") 48))(-> a10-acf
(plotly/layer-bar {:=x "lag"
:=y "acf"
:=title "ACF: Australian antidiabetic drug sales"}))Slow decay (trend) + scalloped shape (seasonality at lag 12, 24, 36...)
A white noise series has no autocorrelation. All ACF spikes should fall within ±1.96/√T.
(def white-noise
(tc/dataset {"t" (range 1 51)
"wn" (repeatedly 50 #(let [u1 (rand) u2 (rand)]
(* (Math/sqrt (* -2 (Math/log u1)))
(Math/cos (* 2 Math/PI u2)))))}))(-> white-noise
(plotly/layer-line {:=x "t"
:=y "wn"
:=title "White noise"}))(def wn-acf (acf (white-noise "wn") 15))(let [bound (/ 1.96 (Math/sqrt 50))]
(-> wn-acf
(tc/add-column "upper" (repeat (tc/row-count wn-acf) bound))
(tc/add-column "lower" (repeat (tc/row-count wn-acf) (- bound)))
(plotly/layer-bar {:=x "lag"
:=y "acf"
:=title "ACF: White noise"})))All spikes should be within ±0.28 (= 1.96/√50) → Confirms: no signal to model.
Two approaches to building seasonal plots with many traces:
:kindly/f, then post-process each traceLet's time them:
(defn seasonal-plot-manual
"Build seasonal Plotly spec manually (no tableplot)."
[ds phase-col value-col group-col color-fn
& {:keys [line-width title x-title y-title]
:or {line-width 0.3}}]
(let [groups (-> ds (tc/group-by group-col) :data)
traces (mapv (fn [group-ds]
(let [group-name (first (group-ds group-col))
sorted-ds (tc/order-by group-ds phase-col)]
{:x (vec (sorted-ds phase-col))
:y (vec (sorted-ds value-col))
:type "scatter"
:mode "lines"
:name group-name
:showlegend false
:line {:color (color-fn group-name)
:width line-width}}))
groups)]
{:data traces
:layout {:title title
:xaxis {:title x-title}
:yaxis {:title y-title}}}))Using the daily seasonal plot (700+ days = 700+ traces)
(let [color-fn #(get {"2011" "#7570b3" "2012" "#1b9e77" "2013" "#d95f02" "2014" "#7570b3"}
(subs % 0 4) "gray")
n 10]
{:tableplot-kindly-f
(let [start (System/nanoTime)]
(dotimes [_ n]
(seasonal-plot-spec vic-elec-with-fields "DailyPhase" "Demand" "TimeDate" color-fn))
(/ (- (System/nanoTime) start) 1e6 n))
:manual-traces
(let [start (System/nanoTime)]
(dotimes [_ n]
(seasonal-plot-manual vic-elec-with-fields "DailyPhase" "Demand" "TimeDate" color-fn))
(/ (- (System/nanoTime) start) 1e6 n))}){:tableplot-kindly-f 1016.6073576000001,
:manual-traces 246.06322740000002}
Kindly is a portable notation protocol for Clojure visualizations. When tableplot builds a plot, it returns a map like:
{:kindly/f (fn [m] )
:other-keys ...}
The :kindly/f function transforms the map into actual Plotly JSON. This defers evaluation — Clay/Portal can choose when to render.
To get the raw spec for post-processing:
(let [viz (plotly/layer-line ...)
spec ((:kindly/f viz) viz)] ;; invoke the function
(update spec :data ...)) ;; now we can modify traces
Why not just return the spec directly? Deferred execution enables:
layer-* calls before computing)| R (fpp3) | Clojure (tablecloth + tableplot) |
|---|---|
autoplot() | plotly/layer-line |
gg_season() | extract year/month + layer-line with :=color |
gg_subseries() | extract month + faceted/colored line plot |
gg_lag() | manual lag column + layer-point |
ACF() | acf function (to be added to tablecloth.time) |
ggplot + geom_* | plotly/layer-point, plotly/layer-bar, etc. |
New function for tablecloth.time: acf — autocorrelation computation. This should live in the column API alongside the field extractors.
source: notebooks/chapter_02_time_series_graphics.clj